Assistant Professor | School of Information Science
N = 27
N = 46
N = 99
N = 147
N = 108
The Cook and Campbell (1963/1966/1979) Framework Shadish, Cook, and Campbell (2002)
Know these names, especially if you are doing experimental work
4 Key Dimensions:
19 QUESTIONS
8 RATING SCALES
Q1: What are the key independent variables (LOM)?
Q2: What are the key dependent variables (LOM)?
Q3: What are the main RQs and/or RHs?
Q4: What is the research approach?
Q5: What is the general design classification (randomized, etc.)?
Q6: What is the specific design/approach?
Q7: Is the measurement reliability for each key variable acceptable?
Q8: Is the evidence of measurement validity for each key variable acceptable?
Q9: What is the overall rating of measurement reliability and statistics?
Q10: Are the comparison groups equivalent (random assignment)?
Q11: What is the evaluation of the control of extraneous experience and environment variables?
What is the evaluation of the . . .
Q12: construct validity of the intervention (i.e., did the manipulation work?
Q13: construct validity of the outcome measures (DVs) and any attribute IVs?
Q14: overall population external validity?
Q15: overall ecological external validity?
Q16: extent to which important participant subgroups were tested and/or compared?
Q17: Was there adequate peer review?
Q18: Do the authors adequately present the case for the theoretical importance or practical relevance of their RQs and design?
Q19: Do the authors interpret their findings adequately? Were the title, abstract, and discussion clear and accurate (as opposed to overstated and misleading) given the evaluation of the several aspects of research validity?
Q1 - Q6
Q7 - Q9, Q12 - Q13
Q10 - Q11
Q14 - Q16
Q17 - Q19
The outcome of research is the GENERATION of more RESEARCH QUESTIONS
Knowledge is power!
Keating, D. M., Richards, A. S., Palomares, N. A., Banas, J. A., Joyce, N., & Rains, S. A. (2022). Titling practices and their implications in communication research 1970-2010: Cutesy cues carry citation consequences. Communication Research, 49(5), 627-648. https://doi.org/10.1177/0093650219887025
“This article examines the 408 articles titled ”Should I Stay or Should I Go?” and asks why there are so many articles with the same title. The academic culture of publication pressure can lead to researchers choosing creative titles, including popular song titles, to stand out from the crowd. Furthermore, risk assessment leads researchers to choose the same songs as others, because well-known, easy-to-understand cultural references are better rewarded than more obscure references. The collective outcome of this, many researchers choosing the same title for their articles, reflects the mass production of cultural products, wherein creativity is standardized and panders to the largest possible audience.”
Nieuwenhuis, J. (2022). Another article titled “Should I Stay or Should I Go?” or, the mass production of academic research titles. The Information Society, 1-7. https://doi.org/10.1080/01972243.2022.2152916
Ultimate purpose is to instruct the reader on exactly what was done to allow them to replicate the study under identical conditions.
There was a [negative or positive] correlation between the two variables, r(df) = [r value], p = [p-value]
EX: There was a negative correlation between the two variables, r(33) = -.37, p = .029.
4 Things to consider with any test:
The key is to make sure limits and future directions are meaningful:
I like to find the source on Google Scholar, use their generator, fix it, then grab the doi.
Can use Appendix F but APA manual is better if you have it (Ch. 7 - pp. 195-250)
Myers, S. A., Edwards, C., Wahl, S.T., & Martin, M.M. (2007). The relationship between perceived aggressive communication and college student involvement. Communication Education, 56, 495-508.
Goldman, Z. W., & Goodboy, A. K. (2014). Making students feel better: Examining the relationships between teacher confirmation and college students’ emotional outcomes. Communication Education, 63(3), 259-277
Busting the mythology of Reviewer 2
If you get to this point, it should be accepted
This is the hidden curriculum. I learned how to do this when I took qualitative methods as a second year PhD student from this same process.
Good researchers support and justify they decisions they made along the way.
After having your studies posted on the CI SONA webpage for at least a week, you should now have some data to begin exploration. This class period will be very applied – demonstrating how to download your data from Qualtrics, organize the database, clean the data, identify outliers, gather descriptive, and prepare everything for analysis.